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Abstract. The Tikhonov regularization of linear ill-posed problems with an 
penalty is considered. We recall results for linear convergence rates and results on 
exact recovery of the support. Moreover, we derive conditions for exact support 
recovery which are especially applicable in the case of ill-posed problems, where 
other conditions, e.g. based on the so-called coherence or the restricted isometry 
property are usually not applicable. The obtained results also show that the 
regularized solutions do not only converge in the £^-norm but also in the vector 
space iP (when considered as the strict inductive limit of the spaces R" as n 
tends to infinity). Additionally, the relations between different conditions for 
exact support recovery and linear convergence rates are investigated. 

With an imaging example from digital holography the applicability of the 
obtained results is illustrated, i.e. that one may check a priori if the experimental 
setup guarantees exact recovery with Tikhonov regularization with sparsity 
constraints. 
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1. Introduction 



In this paper we consider linear inverse problems with a bounded linear operator 
^ : J{i — > J{2 between two separable Hilbert spaces 5£i and 'K^, 



Af = 9. 





We assume that the operator equation Af — g has a solution /* that can be 
expressed sparsely in an orthonormal basis ^' :— {ipiyi^z of !Hi, i.e. /* decomposes 
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into a finite number of basis elements, 

witli u'>ef{Z,R), and \{i e Z\uf ^ 0}\ < oo. 

Tlie knowledge that /* can be expressed sparsely can be utilized for the reconstruction 
by using an £^ -penalized Tikhonov regularization ^T, i.e. an approximate solution is 
given as a minimizer of the functional 

k\\Af~f\\j,.,+aY,\{f,m (2) 

with regularization parameter a > 0. In contrast to the classical Tikhonov functional 
with a quadratic penalty [13] , the ^"'^-penalized functional promotes sparsity since small 
coefficients are penalized more. 

For the sake of notational simplification, we use = €^(Z,]R) and introduce the 
synthesis operator 13 : £^ — )■ IKi, which for m G £^ is defined by Du = 'Y^Uiipi. With 
that and the definition K :— Ao D : £^ ^ 'K2 we can rewrite the inverse problem ([ij 
as Ku = g. Adopting the usual convention in convex analysis we use the following 
somewhat sloppy notation 



IImII^i , if M G i'"'", 

00, \iu(^e\i^, 



and we rewrite the ^-'^-penalized Tikhonov regularization ^ as 

T^{u) := \\\Ku-g'\\li^+a\\u\\,i. (3) 

In the following we frequently use the standard basis of which is denoted by {ej}j^z- 
The Tikhonov functional ^ has also been used in the context of sparse recovery under 
the name Basis Pursuit Denoising [s]. 

Daubechies et al. [7j showed that the minimization of ([s]) is indeed a regularization 
and derived error estimates in a particular wavelet setting. Error estimates and 



convergence rates under different source conditions have been derived by Lorenz 22 
and Grasmair et al. [16| . In this paper we aim at conditions that ensure that the 
minimizers u"'^ € argminTc(u) have the same support as w*. In the context of sparse 
recovery, this phenomenon is called exact recovery. One of the main applications in 
the field of sparse recovery is compressive sampling, a new sampling technique which 
allows to sample sparse signals at low rates [sj. Our approach builds heavily on 



techniques and results from the field of sparse recovery from 12 T4][15|[19|[28] some of 
which we transfer to the field of inverse and ill-posed problems. Very roughly spoken, 
the conditions for exact recovery can be divided into two classes: Sharp conditions 
which are not practical since they rely on unknown quantities (in this category are for 



example Tropp's ERG 28 theorem 8] and the null space property [18]). More loose 
conditions which are far from being necessary but seem more practical (in this category 
are for example conditions using incoherence [11| , [28[ corollary 9] and the restricted 
isometry property [ij). Moreover, the latter conditions are usually not applicable for 
inverse and ill-posed problems (somehow due to arbitrarily small singular values of the 
operator). Hence, we focus on "intermediate" conditions [9 12 and show how they 
can be applied to general inverse and ill-posed problems. Especially we contribute 
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the following point: While it is good to know that exact recovery is possible for some 
regularization parameter, what one really needs is a computable recipe to choose a 
parameter which uses only available information and hence, we especially treat this 
question on the choice of a regularization parameter which guarantees exact recovery. 
The paper is organized as follows. In section [2] we summarize some properties of 



£^-penalized Tikhonov minimizers and recall a stability result from 16 . In section 
|3]we review previous results on exact recovery in the context of sparse recovery and 
we illustrate our contribution. Section [4] contains the main theoretical results of the 
paper. Especially known results on exact recovery conditions are transfered to ill- 
posed problems and we give a parameter choice rule which ensure exact recovery in 
the presence of noise (under appropriate assumptions). One novelty here is, that 
we focus on conditions and a choice rule which can be verified a priori and hence 
are of practical relevance (and not only of theoretical relevance) . In section [5] we 



investigate the relation between the ERG from 28 , the source condition and the null 
space property 18 . In section |6j we demonstrate the practicability of the deduced 



recovery condition with an example from imaging, namely, an example from digital 
holography. In section[7]we give a conclusion on exact recovery conditions for Tikhonov 
regularization with sparsity constraints. 



2. The i!^-penalized Tikhonov functional 

Before we start with error estimates, we recall some basic properties of the ^^-penalized 
Tikhonov functional Tq,. First we repeat a trivial characterization of the minimizer. 

Proposition 2.1 (Optimality condition). Define the set- valued sign function Sign : 

e ^ {{-1}, [-1, +1], {+l}}^ for u e e, by 

[{-1}, Uk<Q, 
(Sign(M)), := <^ [-1,+1], uu^Q, (4) 
[{+1}, Uk>Q. 

Let u"'^ e l"^ . Then the following statements are equivalent: 

(i) m"^^ e argminrQ(M). (5) 

(ii) - K*{Ku'^'' - g') e aSign(M"'=). (6) 

Proof. Since the the set-valued sign function is actually the subgradient of the norm, 
the proof consists of noting that (ii) is just the optimality condition G 9Tq,(w"'^). 
Due to convexity of Tq, (ii) is also sufficient. □ 

Another well known characterization of a minimizer u"'^ is, that it is a fixed point 
of w"'^ — Eia{u°'''^ + K* {g'^ — Ku"''^)), where Sq, denotes the soft-thresholding operator, 
cf. e.g. [7] . From this characterization or from ([6]) we can deduce the following: Since 
the range of K* is contained in any minimizer u"''^ of the ^^-penalized Tikhonov 
functional Tq, is finitely supported for every a > 0. (Note that this observation relies on 
the fact that K is bounded on If we model K : £^ 'K2 boundedly, as appropriate 
for normalized dictionaries, we cannot conclude that u"'^ is finitely supported, since 
the adjoint operator maps K* : 'K2 — > ■) 
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Uniqueness of the minimizer of ([3]) could be guaranteed by ensuring strict 
convexity. This holds, e.g., if K is injective. A weaker property of the operator 
which also guarantees uniqueness (although the functional is not strictly convex) 
is the FBI [T property defined below. 

Definition 2.2. Let K : i"^ 3^2 be an operator mapping into a Hilbert space 
Then K has the finite basis injectivity (FBI) property, if for all finite subsets J d Z the 
operator restricted to spanje^ | z € J} is injective, i.e. for all u, u G £^ with Ku = Kv 
and Uk — Vk — 0, for all k ^ J /it follows that u = v. 



In inverse problems with sparsity constraints the FBI property is used for a couple 
of issues concerning ^-penalized Tikhonov functionals, for example for deduction of 



stability results 16 17,22 , for derivation of efficient minimization schemes 20 21 
and for proving convergence of minimization algorithms [l]|6][2^. A demonstrative 
example for an operator which possesses the FBI property but is not fully injective 
is the following: Denote with {ei}i=o,... the usual real Fourier basis of L^([0, 1]) 
and with {-(Ajlj^i.... the Haar wavelet basis of L^([0, 1]) and define the operator 
K : £2(Z) ^ L^([0,1]) by Ku = J2T=o^J^j + E7=!^ i V'-j • Then K is clearly 
not injective, since any Haar wavelet can be expressed in the Fourier basis and vice 
versa. However, K obeys the FBI property since neither any Haar wavelet is a finite 
linear combination of elements of the Fourier bases nor the other way round. 

The FBI property is related to the so-called restricted isometry property (RIP) [4] 
of a matrix, which is a quite common assumption in the theory of compressive 
sampling [2|[3]. The RIP is defined as follows. Let ^ be a m x n matrix and let 
s < n be an integer. The restricted isometry constant of order s is defined as the 
smallest number < Cs < 1, such that the following condition holds for all v E E" 
with at most s non-zero entries: 

il-cMv\\%<\\Av\\%<{l + Cs)M%. 

Essentially, this property denotes that the matrix is approximately an isometry when 
restricted to small subspaces. The FBI property, however, is defined for operators 
acting on the sequence space and only says, that the restriction to finite dimensional 
subspaces is still injective and makes no assumption of the involved constants. 

With we denote the vector space of all real-valued sequence with only finitely 
many non-zero entries. In contrast to the £p spaces with p > there is no obvious 
(quasi-)norm available which turns i'^ into a (quasi-)Banach space. We will come back 
to the issue of defining a suitable topology on later. In general, the minimum- 1| • ||^i 
solution of Ku = g neither needs to be in nor needs to be unique. If we assume 
that there is a finitely supported solution e of Ku — g, then the set of all 
solutions of Ku = g is given by -I- ker K. If K possesses the FBI property, then the 
solution is the unique solution in £'^, hence ker if C \ However, in general 
zi* G is not a minimum- 1| • ||^i solution. In the following we assume that K possesses 
the FBI property and denote the unique solution of Ku — g in i'^ with . 

Stability and convergence rates results for i'^-penalized Tikhonov functionals have 
been deduced in (7][l6|l7l|22| . The following error estimate from |16| ensures the linear 



convergence to the minimum-|| • jj^i solution, if a certain source condition is satisfied. 
We state it here in full detail and give explicit constants. 
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Theorem 2.3 (Error estimate [TBj theorem 15]). Let K possess the FBI property, 
u* e £^ withsuppu^ — I be a minim,um-\\ -Wii solution of Ku — g, and \\g—g^\\:K2 ^ £• 
Let the following source condition (SC) he fulfilled: 

there exists w S 3^2 such that I'C*w ~ € Sign(u*). (7) 

Moreover, let 

^ = sup{|ai|iai<l} 
and c > such that for all u E l"^ with supp(u) <Z I it holds 

\\Ku\\ > c\\u\\. 

Then for the minimizers u"'*^ of Ta it holds 

Especially, with a >i e it holds 

=0(e). (9) 

Remark 2.4. a) Since £, € hy definition of K, it is clear that < 1. 

b) Since K possesses the FBI property, the existence of c > is ensured. 

c) To achieve the hnear convergence rate (|9|, the SC Q is even necessary, cf. 17 



The above theorem is remarkable since it gives an error estimate for regularization 
with a sparsity constraints with comparably weak conditions of the operator, especially 
nothing is assumed about the incoherence of K in either way. However, the constants 
in the error estimate ([s]) are both depending on the unknown quantities /, and 9 
and are possibly huge (especially c can be small and 6 can be close to one). 

3. Known results from sparse recovery 



In 28 Tropp deduces a condition which ensures exact recovery. To formulate the 
statement, we need the following notations. For a subset J C Z, we denote with 
Pj : £^ — £^ the projection onto spanje; | i S J}, 

i.e. the coefficients j ^ J are set to and hence supp(Pju) C J. With that definition 
KPj : 3-C2 modifies the operator K such that KPju only depends on the entries 
Ui for i £ J and hence is something similar to the restriction of K to spanje^ \ i G J}. 

Moreover, for a linear operator B we denote the pseudoinverse operator by B^ . 
With these definitions we are able to formulate Tropp's condition for exact recovery. 

Theorem 3.1. Let K be bounded and assume that KPj is infective; let gj be the 
orthogonal projection of g^ to the range of KPj and denote with Uj G i"^ the unique 
element with supp(uf) C / such that gj = Kuj. 
If the exact recovery condition (ERG) 

snp \\{K PiY Ke,\Ui <1, (10) 
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holds, then the parameter choice rule 
suPig/C \ {g'' - gj,Ke,) 



l-sup,g,c \\{KPi)^Ke,\\,i 
ensures that supp(u"''^) = supp(?i*). 



<a < 



min \uj{i) 
iei 



\{PiK*KPiy^\,i^,i 



(11) 



This theorem can be extracted from 28 under shghtly different assumptions 
(basically it is theorem 8 there, however, this relies on several other results in [28l). 

The applicability of this result is limited due to several terms: The expressions 
II P/)~^ ll^i £1 and \\{KPi)'^ KciWii , need the knowledge of / which is unknown. 
Moreover, the quantities (the projection of onto the range of KPi) and 
(the unique element with supp(uf) C / such that = Ku'^) are unknown and not 
computable without the knowledge of /. 

In the following section we deduce a-priori parameter rules that are easier to use 



than Tropp's parameter choice rule (11). The idea is to get rid of the expressions g'j 
and u 'y, whic h cannot be estimated a priori. Furthermore, we will apply the techniques 
to deal with the terms \\{PiK* KPi)-'^\\ii^ii and \\{KPi)U<e,\\f,i . 



from 



12 



19 



Finally we remark that there are conditions for exact recovery (also in the presence 



28 



of noise) which use the so called coherence of the dictionary in [14, 15 and 
corollary 9]. The conditions are much easier to check (since they only rely on inner 
products (KeijKej)) but are also much harder to fulfill in practice. 



4. Beyond convergence rates: exact recovery for ill-posed operator 
equations 

In this paragraph we give an a priori parameter rule which ensures that the unknown 
support of the sparse solution € is recovered exactly, i.e. supp(u"'^) — supp(u*). 
We assume that K possesses the FBI property, and hence is the unique solution of 
Ku = g in With / we denote the support of u*, i.e. 

/ supp(w*) {ieZ\uf ^ 0}. 

Theorem 4.1 (Lower bound on a). Let e supp(u*) ~ I, and g^ ~ Ku'' + 77 
the noisy data. Assume that K is hounded and possesses the FBI property. If the 
following condition holds, 

sup||(XP/)tife,||fi< 1, (12) 



then the parameter rule 



a > 



l + sup,g,c \\iKPi)^Ke 



l-sup,g^c \\{KPi)^Ke^\U 
ensures that the support of u"'^ is contained in I. 



sup|(ry, ivTcj 



(13) 



Proof. In [28j theorem 8] it is shown that condition (12) together with the parameter 
rule 

sup,e/c lig" - ghKe,)\ 



a > 



l-sup,g,c \\{KPi)^Ke,\Ui 
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gj = {ld-KPi{KPi)^){g'^g) = {ld-KPi{KPi)^){i^) 



Hence, using standard identities for the pseudo inverse and Holder's inequality, we get 
that for i e it holds that 

|(/-gf,ife,)| = \{{ld-KPi{KPif)f^,Ke,)\ 
- \{r^,{lA-KPi{KPi)^)Kei)\ 
= \{K*i^,{ld-{KPi)^K)e,)p\ 
< \\K*Tj\\e^ \\{ld-iKPi)^K)e,\\ii. 

Since i G and supp((if P/)^i^ei) C / we get 

sup ||e, - {KPiyKeiWei = 1 + sup WiKPiYKe. " 



Hence, using ||iir*77||£=o = supjg^ |(A'*77, e^)] we end in the estimate 



sup I (5'' 



gf,ife,)| < sup I (?7, iCe, 

iGZ 



sup \\{KPi)^Ke 



(14) 



Thus, the condition ( 12 ) together with the parameter choice rule ( 13 1 ensures that the 
support is contained in /. A direct proof without using 28 can be found in 27 . □ 



Remark 4.2. Instead of using the estimate p4| , one can alternatively use another 
more common upper bound for sup^gjc Kff"^ — gj,Kei)\. For that notice that 
Id— KPj{KPjy is the orthogonal projection on Tg{KPi)^. Then, since the norm of 
orthogonal projections is bounded by 1, we can estimate for i e with the Cauchy- 
Schwarz inequality as follows 

\{r,,{ld-KPi{KPiy)Ke,)\ < hl^JI (Id -XPj(XPj)t)i^e,|U, < e\\K\\. (15) 



In general, one cannot say which estimate gives a sharper bound, inequality ( 14 ) or 
inequality ( 15 ). However, in practice the noise ry often is a realization of some random 
variable e.g. with symmetric distribution and hence, supjgg \{ri,Kei)\ <^ £\\K\\ seems 
plausible. In this case the estimate with Holder's inequality ( 14 ) gives a sharper 
estimate and we use (14) for the example from digital holography in section |6] 



Theorem |4.1| gives a lower bound on the regularization parameter a to ensure 
supp(u"''^) C supp(u*). To guarantee supp(u"''^) — supp(u*) we need an additional 
upper bound for a. The following theorem leads to that purpose. 



Theorem 4.3 (Error estimate). Let the assumptions of theorem 4-i hold and choose 
a according to (13). Then the following error estimate is valid: 



< {a + svip\{f],KeM\{PiK*KPi) 



(16) 
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Proof. From the assumptions of theorem 4.1 we have supp(u"'^) C supp(u*). From 
the optimahty condition (|6]) we know that for u°''^ there is a € with ||?«||foo < 1 
such that 

-K*{Ku"'^ - g^) ^aw. 

Hence, it holds that 

-PiK*KPi{u"'' -u'') = -PiK^iKu"-' ~ g) = -PjK* {Ku"-' - g') - PiK*t] 
= aPiw — PiK*rj. 

Since ||?«||f=o < 1, with Holder's inequality we can estimate for all j G / 

- |(w"■^-u^e,)| = \{{PiK* KPi)-\aPiw - PiK*Ti),ej)\ 

< a\{Piw, {PiK*KPi)-^ej)\ + \{PiK*r^, {PiK* KPi)-'e,)\ 

< {a\\w\U^ + \\PiK*^\\e^)\\iPiK*KPi)-'y.ii 

< {a + sup\{r],Ke,)\)\\{PjK*KPi)-'\\,iji. 



Remark 4.4. Due to the error estimate ( 16 1 we achieve a linear convergence rate 
measured in the norm. In finite dimensions the norms are equivalent, hence we 
also get an estimate for the error: 

- < {a + e\\K\\) \I\ \\{PIK*KPI)-'\\l^^,^. 

Compared to the estimate (jsj from theorem 2.3 the quantities 6 and are not 
present anymore. The role of 1/c is now played by \\{PiK* KPj)~^\\i^i . However, 
if upper bounds on / or on its size (together with structural information on K) is 
available, our estimate can give a-priori checkable error estimates. 

The following theorem gives a sufficient condition for the existence of a 
regularization parameter a which provides exact recovery. Due to theorem |4.3[ 
equation (16), the regularization parameter should be chosen as small as possible. 

Theorem 4.5 (Exact recovery condition in the presence of noise). Let e £° with 
supp(u*) = / and g^ — Ku'-' + rj the noisy data with noise-to-signal ratio 

sup 1(77, Xe^) I 

iez 

^''/" I, ,01 ■ 



Assume that the operator K is bounded and possesses the FBI property. Then the 
exact recovery condition in the presence of noise (eERC) 

sup \\{KPi)^Ke,y < 1 - 2r^/J{PiK*KPi)-^\\fi,fi (17) 
ensures that there is a suitable regularization parameter a, 



l + sup^g,c \\{KPi)^Ke,y 
l-sup,g,o ||(ifP/)ti^e,||,i 



sup |(?7, Kci) I < a (18) 
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which provides exact recovery of I, i.e. the support of the minimizer u"'^ coincides 
with supp(u*) = /. 



Proof. The lower bound of the parameter rule (18 1 ensures that supp(u"'^) C /. With 



the error estimate (161 we see that the upper bound from the parameter rule (181 



guarantees for j & I that 



uf^l < min|u,f| 



hence \uf'\ > - miuie/ > for aU j G /. The eERC 
interval of convenient regularization parameters a resulting from 



(17) 



ensures that the 
is not empty. □ 



Remark 4.6. Theorem |4.5| gives a parameter choice rule that works without the 



quantities gf and u'j. In theorem 3.1 the existence of a parameter that ensures exact 



recovery is guaranteed on the condition 



sup||(ifP/)^A'e, 



< 1 



miuig/ \u%i)\ 



\{PjK*KPiy 



(19) 



cf. equation (111. The question remains which condition is sharper, (19) or (18)? The 



conditions look similar with different factors on the right-hand side, however, it is not 
obvious, which condition is shaper: Indeed, provided that supj^^^c \\{KPjy Kei\\(i < 1 
holds, by using equation (14) we can estimate 



sup.g/c \{g' - .gf , Ke,)\ ^ ^ sup^g^ \{r], Ke 



min^g/ \u]{i)\ 



miuig/ \u]{i)\ 



However, it is not clear which expression, miuig/ or minig/|u*|, is smaller: 
Recall, (/f is the orthogonal projection of to the range of KPj, and Uj is the unique 
element with supp(uf ) C / such that gj = Kuj. Now, the noise r/ with g^ = i^u* + rj 
can cause an increase or decrease of the values Uj and hence, the coefficients uj(j), 
j € I can be larger or smaller. 



The results of theorem 4.3 and 4.5 can be rephrased as follows: If the 
regularization parameter a(£) is chosen according to (18) and fulfills axe, then 
_ yOjl^i = 0{e) and the support of u"'^ coincides with that of u*. Indeed, 
this can be interpreted as convergence in the space with respect to the following 
topology. 



Definition 4.7. We equip the spaces M" with the Euclidean topology and consider 
them ordered by inclusion M" C M""'""'^ in the natural way. Then an absolutely convex 
and absorbent subset U of £° is called a neighborhood of if set [/ n M" is open in 
R" for any n. The topology t on which is generated by the local base of these 
neighborhoods is called the topology of sparse convergence. 

The definition above says that the topology of sparse convergence is generated as 
strict inductive limit of the spaces K". The space £° is turned into a complete locally 
convex vector space with this topology. A sequence (m") in converges to u in this 
topology if there is a finite set / C N such that suppu" C /, for all n G N, and the 
sequence m" converges componentwise. As a strict inductive limit of Frechet spaces, 
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(^°,t) is also called an LF-space and is known to be not normable, see 10 23 . The 
topology of sparse convergence resembles the topology on the space of test functions 
T>{Vt) in distribution theory. (This correspondence can be pushed a little bit further 
by observing that the dual space of r) is the space of all real valued sequences and 
plays the rule of the space of distributions. We will not pursue this similarity further 
here.) 



Corollary 4.8 (Convergence in l^). In the situation of theorem 4-5 assume that a is 
chosen to fulfill the parameter choice rule (18 I. Then u"'^ — > in in the topology 
of sparse convergence. 



In fact, the parameter choice rule ( 18 1 is not an a priori parameter rule a = a{e), 



since it depends on the noise rj and on unknown quantities such as / and miuj |it* 



However, the term supjg; 
by ell-R^II, cf. remark 4.2 



I {rj, Kei) \ is related to the noise level and it can be estimated 
The term min^ i.e. the smallest non-zero entry in the 
unknown solution, may be estimated from below in several applications. Due to the 
expressions \\[PiK* KPi)''^\ii and sup^gjc \\{KPi)'^ KaWti , the eERC ^ is hard 



to evaluate, especially since the support / is unknown. Therefore, we follow [12] and 
give another sufficient recovery condition which is on the one hand weaker in the sense 
that it is easier to satisfy (and implies the eERC and hence, is less powerful than the 
ERC) and on the other hand is easier to evaluate in practice since it only depends 
on inner products of images of K restricted to / and For the sake of an easier 



presentation we define according to 12 , 19 



COR/ := s\xv^\{Kei,Kej)\ and COR^o := sup ^ K/sTci, iCej) | 



3.^1 



Theorem 4.9 (Neumann exact recovery condition in the presence of noise). Let 
€ with supp(u*) ~ I and g^ — Ku^ + rj the noisy data with noise-to- signal 
ratio r^i/u- Assume that the operator norm of K is bounded by 1 and that K possesses 
the FBI property. Then the Neumann exact recovery condition in the presence of noise 
(Neumann eERC) 



COR/ + COR^c < niin||ii:e,;||^^ - 2r^/„ 



(20) 



ensures that there is a suitable regularization parameter a, 
COR/ + COR^c 



min 1 1 ivTcj 11^ 



min||i^ej||^ 



COR/ 

a < ( 



— — — sup \{r],Ke^ 



< a 



(21) 



min \ \Ke. 
iei 



iWK2 - COR/) min|M*| - sup |(?7, isTci 



which provides exact recovery of I , i.e. the support ofu'^''^ coincides wit/i supp(u*) — I. 



Proof. For the deduction of conditions (20) and (21) from conditions (17) and (18), 



respectively, one splits the operator PiK*KPi into diagonal and off-diagonal and uses 
\\K\\ < 1 and a Neumann series expansion for its inverse, following the techniques 
from fT2l[T9l. □ 
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Remark 4.10. By the assumptions of theorem |4.9[ the operator norm of K is 
bounded by 1, i.e. ||Xei||M2 — 1 fo'" « G Z. Hence, to ensure the Neumann 



eERC (20), one has necessarily for the noise-to-signal ratio r^/„ < 1/2. For a lot of 
examples one can normalize K, so that Ij^ftTeiUj^j = 1 holds for all i G Z. We do 
this for the example from digital holography in section [6j In this case the Neumann 



eERC (|20|) reads as 

COR/ + CORjc < 1 - 2r^/„. 

This condition coincides with the result presented in [Oj for the orthogonal matching 
pursuit. 

Remark 4.11. We remark that the correlations COR/ and CORjo can be estimated 
from above, with N :— ||m*||£0, by 

COR/ <{N -1)^1 and COR/c < N ^i. 

Consequently, the exact recovery condition in terms of the coherence parameter /i 



recovery condition from 15 



can easily be deduced from conditions (20 1 and (21 1. This will result in Fuchs' exact 



5. Relations between recovery conditions and the source condition 

In this section we compare the different conditions which have been used. As we 
have seen in theorems 2.3 and 4.3 both the SC ([t]) and the ERC (10 1 lead to a linear 



convergence rate under an appropriate parameter choice rule. However, the latter 
also leads to exact recovery. One should note that the ERC and the SC are crucially 
different is some sense: The ERC is a uniform condition in the sense that it uses a 
given support / and hence, also leads to a result which holds for all vectors with that 
support. The SC on the other hand depends on a particular sign pattern Sign(u*) and 
hence, leads to a result which holds for all vectors with that sign pattern. We may 
hence strengthen the SC to a "uniform source condition" (uniform SC) as follows: 

for all with suppu* C / there exists w G 5^2 such that K*w G Sign(?i*). 

As it turns out, the ERC does not only imply the uniform SC but even a "uniform 
strict source condition": 

Proposition 5.1 (ERC uniform strict SC). Let I be finite and let KPj be injective. 
Then the ERC ( [lO| implies the following uniform strict SC; 

PiK*w = Pi sign(u*) 
\\PjnK*w\U^ < 1. 

(22) 



for all with supp(u*) C / there exists w G ^2 



Proof. Let be such that supp(u*) C /. Since KPj is injective, the operator 
PjK* : 3-C2 ^^{I) is surjective and hence, the equation 

PiK*w = P7sign(M*) 

has a solution which can be expressed as 

w = {PiK*yPiK*w = (P/if*)^P7sign(u*). 
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Now it remains to check, that for j ^ I it holds that \ {K*w, ej)\ < 1: With the Holder 
inequality and the ERC it follows that 

\{K*w,e,)\ = \{K*{PjK*yPjsign{uO),e,)\ 
= \{Pjsign{u^),{KPj)^Ke,)\ 
< \\Pisign{u^)\U^\\{KPiyKe,\\,i < 1. 

Finally, \{K*w,ej) \ — ^ for j — > oo and hence \\PjaK*w\\i<=c < 1. □ 



It should be noted that a similar result appears in 17 Theorem 4.7]. There it is 
shown that a linear convergence rate for the minimizers u"'^ already implies that the 
strict SC holds and hence, by theorem 4.3 ERC implies strict SC. 

Another important condition in the context of sparse recovery is the so called 
null space property (NSP). An operator K : l'^ 'K2 is said to have the NSP for the 
set J C N, if for any u € ker u it holds that 

||P,?/||,i < ||P^c?/||,i. (23) 

The importance of the NSP comes from the following theorem on the performance of 



Theorem 5.2 ( [Ts] Thm. 2, Thm. 3]). Any vector u'^ with suppw* C I is the unique 
solution of 

min||ii||^i s.t. Ku — Ku'^ 

u 

if and only if K fulfills the NSP for the set /. 

However, the NSP is implied by the uniform strict SC: 



Proposition 5.3. The uniform strict SC (22) implies the NSP (23). 

Proof. For any u G ker K and any v E 'K2 it holds that 

{Ku,v) ^ {u,K*v). 

Now we define it* by 

i € I : sign(uf ) = - sign(ui), i ^ I : uf = 0. 



Due to (22) we can find w such that K*w € Sign(u*) and moreover ||PfCw||£°o < 1. 
Using this w instead of u, we get from the definition of and the Holder inequality 

= {u,K*w) = ^u^sign('u*) + ^u^{K*w)^ < + ||P/Cu||£i 

which shows the assertion. □ 

The fact that the strict SC is an important condition in this context was already 
observed in [4| Section II]. Combining their argumentation there with theorem 5.2 
one obtains another proof of proposition |5.3| 



Since there are plenty of conditions which are related to the performance of 
minimization, we end this section with an illustration of the implications between 
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different conditions in the context of this paper. First, the obvious imphcation between 
the different "ERCs" : 

eERC 

Neumann eERC ERC 
Neumann ERC 
And then the relation of ERC, SC and NSP: 



ERC + KPi injective ^ uniform strict SC 



SC 



NSP 



Remark 5.4. The implication "ERC + KPi injective NSP" has already been 
observed in |18j remark 4]. Moreover, |18j example 1] shows that the converse 
implication does not hold. We postpone further investigation of converse implications 
to future work. 



6. Application of exact recovery conditions to digital holography 



To apply the Neumann eERC (20), one has to know the support /. In this case, there 
would be no need to apply complex reconstruction methods. One may just solve the 
restricted least squares problem. For deconvolution problems, however, with a certain 



prior knowledge, it is possible to evaluate the Neumann eERC (20 1 a priori, especially 
when the support / is not known exactly. 



In the following we use the Neumann eERC (20) exemplarily for an inverse 
convolution problem as it is used in digital holography of particles [8 25 . The 
presentation relies on [9] and we reproduce it here for the sake of completeness in 
a compact style. In digital holography, the hologram corresponds to the diffraction 
patterns of the illuminated particles. The hologram is recorded digitally on a charge- 
coupled device (CCD), from the diffraction patterns the size and the distribution of 
particles are reconstructed. 

We consider the case of spherical particles, which is of significant interest in 
applications such as fluid mechanics. We model the particles j G {1, . . . , iV} as opaque 
disks Bj.{- — Xj,- — yj,- — Zj) with center {xj,yj,Zj) e M"^ and radius r. Hence the 
source /* is given as a sum of characteristic functions 

N N 

/* = E "1 XbA- -Xjr- ■ - Zj) =: J2 < Xj- 

The real values it| are amplitude factors of the diffraction pattern that in practice 
depend on experimental parameters. 

The forward operator K : £^ ^ L^(IR^), which maps the coefficients Uj to the 
corresponding digital hologram, is well modeled by a bidimensional convolution * with 
respect to {x,y). In the following t represents the imaginary unit. Let h^^ constitute 
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the Fresnel function defined by 

hzj{x,y) = ^r^exp (iT^||i?|p), with R := {x,y). 

LAZj V / 

With that, the hologram of a particle at position (xj^yj^Zj) and hence the 



corresponding operator response Kcj has the following form 25 



2 

{Kej){x,y) -.^ ^XbA^ " Xj,y ~ yj) * Rc{h^.{x - Xj,y - y^)) . (24) 

The factor 2/(7rr^) assures Kej to be unit-normed, cf. [9j. 

The first step to evaluate the Neumann eERC pO| ) is to calculate the correlation 
\{Kei, Kej)\ with distance gj^i :— {xj—Xi^yj—yi). In the following we assume that all 
particles are located in a plane parallel to the detector, i.e. z := Zi is constant for all i. 
In [9] it has been shown that the correlation in digital holography can be estimated by 
the following majorizing function : — M, with known constants 6l « 0.6748, 
cl ~ 0.7857 and C{d) denoting the area of the intersection of two circles with radius 
r and distance d 

\(Kei,Kej)\(ej.i) < M(\\gj,i\) 

-^ + i""-.{'i-i(|^)'ii^«r"}»'»{i-vii''"""}' (^^' 

which is monotonically decreasing in cf. 

With the estimate (25), we come to a resolution bound for droplets jet 



reconstruction, as e.g. used in 25 . Here monodisperse droplets (i.e. they have the 
same size, shape and mass) were generated and emitted on a strait line parallel to the 
detector plane. This configuration eases the computation of the eNeumann ERC. We 
define that the particles are located at some grid points 

AZ {i e Z|i/A e Z}, 

where the parameter A describes the grid refinement. Assume that the particles have 
the minimal distance 

min \\qj4 G AN, 

i,jfEsupp('uv ) 

then the sums of correlations COR/ and CORjc can be estimated from above. W.l.o.g. 
we fix one particle at the origin and estimate with the worst case that the other 
particles appear at a distance of to the origin, with — [iV/2j < j < [N/2\. Then, 
for p > A we get 

L7V/2J 

COR/ = supJ2\iKe^,Ke,)\ < 2 ^ X(jp), 

LJV/2J 

COR/c = sup V|(ifei,ii:ej)| < sup V M{\jp-i\). 



Consequently, we can formulate an estimate for the Neumann eERC ( 20 1 . 
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Proposition 6.1 (Neumann eERC for Fresnel-convolved characteristic functions). 
An estimate from above for the Neumann eERC 12C^ for characteristic functions 
convolved with the real part of the Fresnel kernel is for p > A 



lN/2i LJV/2J 

2 Mjp) + sup ^(\^P - < 1 - 2r^/- 



(26) 



This means, that there is a regularization parameter a which allows exact recovery 
of the support with the £^ -penalized Tikhonov regularization, if the above condition is 
fulfilled. 



Remark 6.2. In comparison, the exact recovery condition in terms of the coherence 
parameter fi = sup^^j \{Kei, Kej)\{Qj^i) < sup^^^^- A^(||gij,i||) = A^(A) according to 
Fuchs' exact recovery condition from [l5] appears as 

(2iV-l)7W(A) < l-2r,/„, 



see remark 4.11 This condition is significantly worse than (26 1, since asymptotically 

\9j4) \\ejA\~^- 



it holds that M ( 



Condition ( 26 ) of proposition 6.1 seems not to be easy to handle due to the upper 



bound Ai from (25). However, in practice all parameters are known, and one can 
compute a bound via approaching from large p. As soon as the sum is smaller than 
1 — 2r^/„, it is guaranteed that the ^^-penalized Tikhonov regularization can recover 
exactly. 



We apply the Neumann eERC ( 26 ) to simulated data of droplets jets. For the 



simulation we use a red laser of wavelength A — 0.6328]jm and a distance of z = 200mm 
from the camera. The particles have a diameter of lOOjini and for the corresponding 
grid we choose a refinement of 25]jm. Those parameters correspond to that of the 



experimental setup used in 25 26 



After applying the digital holography model, we add Gaussian noise of different 
noise levels and in each case of zero mean. For the coefficients , we choose a setting 
which implies ~ 10 for all i £ I. Figure [l] and [2] show simulated holograms 
with different distances p and different noise-to-signal ratios For all noisy 

examples in the right columns of figure [T] and figure [2] it manually was possible to 
find a regularization parameter a so that all the particles were recovered exactly. 
For minimization of the Tikhonov functional we used the iterated soft-thresholding 
algorithm [?]. 

However, only for the image in figure [l] (p w 721iim) condition ^ of 



proposition |6.1| holds, hence the existence of a suitable regularization parameter was 
guaranteed. For the examples in figure [2] the existence of a regularization parameter 
which ensures exact recovery cannot be shown, i.e. condition (26) of proposition 6.1 



is not valid. In the image on top of figure [2j the particles have a too small distance to 
each other {p w 360iim), and even for the noiseless case condition (26) is not fulfilled. 
The image at the bottom of figurc[2](p « 721]im) was manipulated with unrealistically 
huge noise, so that condition (p6|) is violated, too. 
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Figure 1. Simulated holograms of spherical particles. On the left-hand side the 
noiseless signal is displayed. For reconstruction, the noisy signal on the right- 
hand side is used. The dots correspond to the true location of the particles. The 
existence of a suitable regularization parameter is guaranteed by condition ( |26| 
of proposition |6.l| and hence the ^^-penalized Tikhonov regularization recovered 
all particles exactly. 



7. Conclusion 



With the papers 16 and 17 , the analysis of a priori parameter rules for .^^-penalized 
Tikhonov functionals seemed completed. On the common parameter rule axe, linear, 
i.e. best possible, convergence is guaranteed. In this paper we have gone beyond this 
question by presenting a parameter rule which ensures exact recovery of the unknown 
support of S Moreover, on that condition we achieve a linear convergence rate 
measured in the £^ norm, that comes with a-priori checkable error constants which are 



easier to handle than the ones from 16 . A side product of our analysis is the proof 
of convergence in £^ in the topology of sparse convergence. 

Section [5] analyzes some implications between different condition for exact 
recovery. However, in most cases it remains open whether the reverse implications 
also hold and we postpone this investigation to future work. 



Granted, to apply the Neumann eERC ( 20 1 and the Neumann parameter rule ( 21 1 
one has to know the support /. However, with a certain prior knowledge the 
correlations 



COR/ := sup^ |(i^e„is:ej)| and CORjc := sup ^ Kife^, 



can be estimated from above a priori, especially when the support / is not known 
exactly. That way it is possible to obtain a priori computable conditions for exact 
recovery. In section [6] it has be done exemplarily for characteristic functions convolved 
with a Fresnel function. This shows the practical relevance of the condition. 
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